Robust indoor speaker recognition in a network of audio and video sensors

AbstractSituational awareness is achieved naturally by the human senses of sight and hearing in combination. Automatic scene understanding aims at replicating this human ability using microphones and cameras in cooperation. In this paper, audio and video signals are fused and integrated at different levels of semantic abstractions. We detect and track a speaker who is relatively unconstrained, i.e., free to move indoors within an area larger than the comparable reported work, which is usually limited to round table meetings. The system is relatively simple: consisting of just 4 microphone pairs and a single camera. Results show that the overall multimodal tracker is more reliable than single modality systems, tolerating large occlusions and cross-talk. System evaluation is performed on both single and multi-modality tracking. The performance improvement given by the audio–video integration and fusion is quantified in terms of tracking precision and accuracy as well as speaker diarisation error rate and precision–recall (recognition). Improvements vs. the closest works are evaluated: 56% sound source localisation computational cost over an audio only system, 8% speaker diarisation error rate over an audio only speaker recognition unit and 36% on the precision–recall metric over an audio–video dominant speaker recognition method

Performance Evaluation of Simultaneous Sensor Registration and Object Tracking Algorithm

Reliable object tracking with multiple sensors requires that sensors are registered correctly with respect to each other. When an environment is Global Navigation Satellite System (GNSS) denied or limited – such as underwater, or in hostile regions – this task is more challenging. This paper performs uncertainty quantification on a simultaneous tracking and registration algorithm for sensor networks that does not require access to a GNSS. The method uses a particle filter combined with a bank of augmented state extended Kalman filters (EKFs). The particles represent hypotheses of registration errors between sensors, with associated weights. The EKFs are responsible for the tracking procedure and for contributing to particle state and weight updates. This is achieved through the evaluation of a likelihood. Registration errors in this paper are spatial, orientation, and temporal biases: seven distinct sensor errors are estimated alongside the tracking procedure. Monte Carlo trials are conducted for the uncertainty quantification. Since performance of particle filters is dependent on initialisation, a comparison is made between more and less favourable particle (hypothesis) initialisation. The results demonstrate the importance of initialisation, and the method is shown to perform well in tracking a fast (marginally sub-sonic) object following a bow-like trajectory (mimicking a representative scenario). Final results show the algorithm is capable of achieving angular bias estimation error of 0.0034 o , temporal bias estimation error of 0.0067 s, and spatial error of 0.021m

A Gaussian Process Regression based Dynamical Models Learning Algorithm for Target Tracking

Maneuvering target tracking is a challenging problem for sensor systems because of the unpredictability of the targets' motions. This paper proposes a novel data-driven method for learning the dynamical motion model of a target. Non-parametric Gaussian process regression (GPR) is used to learn a target's naturally shift invariant motion (NSIM) behavior, which is translationally invariant and does not need to be constantly updated as the target moves. The learned Gaussian processes (GPs) can be applied to track targets within different surveillance regions from the surveillance region of the training data by being incorporated into the particle filter (PF) implementation. The performance of our proposed approach is evaluated over different maneuvering scenarios by being compared with commonly used interacting multiple model (IMM)-PF methods and provides around $90\%$ performance improvement for a multi-target tracking (MTT) highly maneuvering scenario.Comment: 11 pages, 10 figure

A Gaussian process based method for multiple model tracking

Manoeuvring target tracking faces the challenge caused by the target motion model uncertainty, i.e., unknown model types or uncertain model parameters. Multiple-model (MM) methods have been generally considered to deal with this challenge, in which a bank of elemental filters is run simultaneously to provide a joint decision and estimation of motion model and localisation. However, if the uncertainty of the target trajectory increases, such as the target moves under mixed manoeuvring behaviours with time-varying parameters, more filters with different model assumptions have to be taken into account to match the motion of the target, which may lead to prohibitive computational complexity. To address this problem, we establish a training based algorithm which can learn the actual motion model as a Gaussian process (GP) regression. Then, by integrating the trained GP into the particle filter (PF), a GP-PF based tracking method is developed to track the manoeuvring targets in real-Time. Monte Carlo experiments show that the proposed method had much lower tracking root mean square error (RMSE) and robustness compared with the most commonly used MM methods